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ABSTBACT - 

This paper describes an application of Computer 
Assisted Instruction named "genpbs^ consisting of five modules of 
preliminary instruction and 19 modules of problems fox students ip an 
introductory genetics course. Eac}i problem module consists of an 
average of 14 problems arranged in increasing level of Q^fficulty. 
The program also contains a report module designed for use by the 
course instructor to obtain an on--line summary of student performance 
in the problem modules. The program was tested by a group of 135 
students in a formal course, and student performance data on the 
problem modules are reported. An evaluation of the program by the 
students indicated a jiighly favorable student reaction and an 
overwhelming preference for the computer problems as compared to 
problems and solutions placed on library reserve. A slight, positive 
correlation vas found between time spent by the students on the 
computer and the course numerical grade. Xhe mean course numerical 
grade for students using the computer was found to be slightly higher 
than the corresponding mean from a previous quarter when the computer 
was not used, but the difference was not statistically significant. 
(Author) 
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THE USE OF CAI TO PROVIDE PROBLEMS FOR STUDENTS 
IN INTRODUCTORY GENETICS^ 

Russell V. Skavaril^. C. William Birky, h?, 
Richard £. Duhrkopf^, and Jeffrey A. Knight^ 
The Ohio State University 

This paper describes an application of Computer Assisted Instruction named 'genpbs* consisting 
of 5 modules of preliminary instruction and 19 modules of problems for students in an introductory 
genetics course. Each problem module consists of an average of 24 problems arranged in increasing 
level of difficulty. The program also contains a report module designed for use by tne course instructor 
to obtain an on-line summary of student performance in the problem modules. The program was 
tested by a group of 135 students in a formal course, and student performance data on the problem 
modules are reported. An evaluation of the program by the students indicated a highly favorable 
student reaction and an overwhelming preference for the computer problems as compared to prob- 
lems and solutions placed on library reserve. A slight, positive correlation was found between time 
spent by the students on the computer and the course numerical grade. The mean course numerical 
grade for students using the computer was found to be slightly higher than the corresponding mean 
from a pre\'ious quarter when the computer was not used, but the difference was not statistically 
significant. 



It is generally agreed that, in a course in genetics, 
a student's ability to work problems correctly is essen- 
tial to a mastery of the subject material. Virtually 
all texts in genetics include problems; see, for example. 
Burns (1972) and Strickburger (1968) which are 
used in Genetics 140 and Genetics 500 at The Ohio 
State University, Srb, uL (1965), Goodenough and 
Levine (1974), Crow (1966), and Singleton (1967). 

Problems are valuable teaching aids because teach- 
ing genetics involves, essentially, the presentation of 
a body of facts: and mastery of the body of facts is 
achieved by the student being able to use the facts in 
real life problem solving. Students learn genetics by 
working problems. Unfortunately, most texts do not. 
include the solutions to all problems; when solutions 
are included, there is no attempt to lead the student 
to the correct answer by identifying a mistake, 
reminding the student of a principle which he has for- 
gotten, etc. 

Instructors of genetics traditionally prepare fully 
annotated problems and solutions. Copies of the";e 
materials may then be distributed to the class, but this 

^Requests tor reprints should be sent to Dr. Skavaril or Dr. Birky. 

' ^Dr. Kussell V. Skavaril ;ind Dr. C. William Birky. Jr. are Professors. 
Department of Genetics. The Ohio State University. 484 West 1 2th Avenue, 
Columbus. Ohio 43210. 

* ' Mr. Richard F. Duhrkopt and Mr. JctTrey A. Knight are Graduate 
Teaching and Research Associates, respectively. Department of Genetics. 
The Ohio State University. 484 West 12th Avenue. Columbus. Ohio *<32lO. 



becomes time-consuming and expensive for a large 
v'lass. Alternatively, sets of problems and solutions 
are put on reserve in a library. Library problem sets 
are, however, difficult to maintain and preserve in the 
numbers needed for effective use by students, par- 
ticularly for large enrollment courses. Even if such 
technical problems can be overcome, library problem 
sets cannot provide immediate feedback to a student 
who is laboring under a misconception of some genetic 
principle, nor can such problem sets avoid presenting 
remedial material to a student who does not need it. 

CAI problems, on the other hand, can obviate most 
of these difficulties. The materials are easy to main- 
tain, preserve, and update. The number of computer 
terminals available to students at our institution ex- 
ceeds the number of library reserve problem sets which 
can reasonably, be provided. CAI can provide immedi- 
ate guidance to a student who is making a minor 
error or who has overlooked a fundamental principle. 
This capability is the principle advantage of CAI. 

In addition, CAI provides individualized instruction 
in that each student may proceed through the material 
at a pace that is consistent with his abilities and time 
commitments. Each student may spend as much time 
as he likes on difficult material and proceed quickly 
over material which he fmds easy or familiar. 

The purpose of this paper is to describe a CAI 
application consisting of problems for students in a 
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genetics course and to report an evaluation of student 
performance on the modules. 

COMPUTER SYSTEM AND FACIUTIES 

CAI at The Ohio State University is run on the IBM 
370/158 central computer of the University Systems 
Computer Center (USCC). There are over 100 remote 



CAI terminals distributed across the campus, the ma- 
jority of which are IBM 2741 Communication Termi- 
nals. Approximately half of the terminals are restricted, 
and the remaining are public terminals. Members 
of the university community use the terminals and the 
central computer of USCC under a time sharing en- 
vironment (multi-programming with a fixed number of 
tasks). 



Table 1. Computer Assisted Instruction Problem Modules in 'genpbs' 



Name 



Number of 
Problems 
Available 



Brief 
Description 



Estimated 

Time 
Required 



mendll 28 MENDL I - Law of segregation; one-factor crosses and pedigrees; probability. 60 

mendl2 23 MENDL II - Law of independent assortment; crosses with two or more factors, 60 
all unlinked. 

Stat 15 STATISTICS - Chi-square. 40 

quant 26 QUANTITATIVE - Quantitative and multiple-factor inheritance. . 60 

link 43 LINKAGE - In higher organisms: sex linkage, mapping, assigning genes to 60 

chromosomes. 

genenv 17 GENE/ENVIRONMENT - Dominance, epistasis, modifiers, background genome, 60 

other interactions between genes; lethality, penetrance, expressivity, other inter- 
actions between gene and environment; twin studies. 

chmthy 26 CHROMOSOME THEORY - - Mitosis; meiosis; and relationship to Mendel/s 30 

laws. 

tetanl 21 TETRADS - Tetrad analysis, other aspects of eucaryotic microbial transmission 30 

genetics. 

chmno 23 CHROMOSOME NUMBER - Variations in number; ploidy. 30 

chiTistr 26 CHROMOSOME STRUCTURE - Variations: deletions, duplications, inversions, 40 

translocations. 

prokar 35 PROKARYOTES - Reproduction and transmission genetics of prokaryotes 60 

art d viruses. 

bchemg 21 BIOCHEMICAL GENETICS - One gene-one enzyme theory; pathways; auxo- 45 

tropic mutants. 

dnal 20 DN A I ** DNA structure, replication, mutation, repair, recombination. 30 

dna2 25 DNA II - Coding; gene fine structure. 45 

prosyn 22 PROTEIN SYNTHESIS Protein synthesis. 30 

devgen 17 DEVELOPMENT GENETICS - Development genetics; cell differentiation: 30 

control of gene action (transcription). 

iichmg 20 • NON-CHROMOSOMAL GENETICS - Genetics of genes in chloroplasts, mito- 40 

chondria, symbionts. 

P'Opgnl 24 POPULATION GENETICS I - Gene pools and frequencies; Hardy-Weinberg 30 

law. 

popgn2 25 POPULATION GENETICS II - Evolution in gene and genotype frequency. 45 
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Abbreviations: OA = Correct A^nswer; WA = Wrong Answer, AA Anticipated Answer; UN = Unanticipated Answer. 
Figure 1 . Problem module logic as used in *genpbs\ 



Description of CAI Modules 

A CAI course named *genpbs'^ has been designed 
and Implemented during 1975. The course was written 
in Coursewriter III, Version 3 (IBM/ 1971, 1973a. 
1973b) and consists of 5 modules of preliminary in- 
struction, 19 modules of genetic problems, a report 
module, and a question skeleton generating macro. 

The five modules of preliminary instruction deal 
with such CAI system features as the completion of 
registration of students on the course, the signal 
for student response or sign off, the use of the *go to' 
commar:! the use of the 'calc functiori, and how a 
student rriay send comments to authors while working 
on the course. 

Table 1 gives the module names, the number of 
problems available, a brief description of the type of 
problem, and an estimate of the time in minutes re- 
quired to complete the module. The categories corre- 
spond to blocks of lectures given in Genetics 500 

♦Arrangements can be made to supply ihe course lo members of ihe cduea* 
lional community. Individuals inierested acquiring *j:enpbs' should et>niact 
Dr. Skavaril. 



(General Genetics) at The Ohio State University, 
but are applicable to virtually any introductory course 
in genetics. Typically, students in Genetics 500 are 
distributed, relative to rank, as follows: 40% luniors, 
30% Seniors, 10% Sophomores, 10% Graduate Stu- 
dents, and 10% Continuing Euncation and other cate- 
gories. Approximately 80% of the students are from 
departments within the College of Biological Sciences, 
as follows: 32% Zoology, 17% Biology, 17% Micro- 
biology, 5% Botany, 6% Genetics, 4% Biochemistry, 
and 1% Biophysics. 

Figure 1 is the flowchart using standard flowchart 
symbols for the logic which is used within each prob- 
lem module of *genpbs'. After a student has completed 
the preliminary instruction modules, he uses the *go 
fo* command to take him to the beginning of the prob- 
lem module (the block labelled *'START" in Figure 
1), ai which point the student is provided with a brief 
description of the nature of the problems in the module 
and asked whether or not he wishes to work the prob- 
lems in the module. If the student enters a negative 
response, he is asked to use the *go to* command 
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to take him to the module on which he wishes to work 
or to sign off the course. If the student responds 
that he does wish to work problems within the module, 
then the first problem is presented to the student and 
the performance data for that module are decoded 
and prepared to be updated in the event that the stu- 
dent decides to answer the question. We have at- 
tempted to write and design each problem so that the 
computer will help the student who is making an error 
to achieve the correct answer to the problem. This is 
done by reminding the student of a fundamental prin- 
ciple after the student has entered a wrong answer and 
asking the student to attempt the problem again, etc. 
In addition, the problems within a module are pre- 
sented to the student in approximate order of increa- 
sing difficulty so that the student has the opportunity, 
when working in a difficult problem, to utilize what he 
has learned by working the preceding, relatively 
simpler problems. 

We have designed a successive wrong answer toler- 
ance parameter which we initially set to six. While 
the student is working on 'genpbs\ the computer 
keeps a count of the number of successive wrong an- 
swers entered by the student. If this count should ex- 
ceed the successive wrong answer tolerance, the stu- 
dent is immediately informed that he has incorrectly 
answered that many questions in a row and is forced 
otT the course. He is informed that the computer will 
not allow him further use of the course until after his 
instructor has authorized additional use. This suc- 
cessive wrong answer tolerance has been incorporated 
as a feature of 'genpbs' so as to encourage .students 
to be serious when they work the problems and to force 
students to obtain help from the instructor and 
teaching associates when their difficulties- exceed the 
computer's instructional capabilities. The count of 
each student's number of successive wrong answers 
is set to zero each time the student enters a correct 
response. The instructor of the course and the grad- 
uate teaching associates have been trained in the on- 
line procedures necessary to allow a student who has 
been forced off the course to resume use of *genpbs*' 
and, if necessary, to change the successive wrong an- 
swer tolerance for a particular student. 

Students who have been forced off the course are 



'Kiauthi>ri/;itlo»» Inr usl* nl *>:i:n|il»\' ttir a sUuk'ni uhn Jias Inrn torcctl olT 
the vourvc is ;tLhiL;L(i hy ihc ihstniLMdr pL-rforniln^ an nn-Iinc aujliur Uiutl ot 
0 inio paratuLter niiu" i>l ihc stutk-nt's rccunl. ll'a sliidcnt whu has Uivn tnrCfd 
lilt ihc oMtirsL- attL-nipts to sif^n on whhoitt tlu* parameter hcin^ changed. Ihi* 
iiuiipiucr will tell Ihc sUnk-nf thai. ai'Lordin^ ti» the cinnptitcr's rtcttrtl, turthvr 
iiM* ul 'j^cnpks' has hvcu rcatuhori/cd; atut l^u.* ciimiyutLr will sif-n such a 
siiictcnt ott thf coiirst.vat:ain. 
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identified in a monitor's student status listing. Stu- 
dents who have attempted to sign on the course with- 
out proper reauthorization are also identified in such 
listings. 

If the student's answer to a question is correct, the 
student's performance record is updated and stored. 
The student is told that his answer is correct, the num- 
ber of successive wrong answers is set to zero, and the 
next problem in the module is presented. If the stu- 
dent's answer was one of the specific wrong answers 
anticipated for the problem, the student is told that 
his answer was incorrect and is given the correct answer 
to the problem. The number of successive wrong an- 
swers is increased by one, and the student is either sent 
on to the nt%t problem in the module or forced off 
the course. 

If the student's response is one of the specific an- 
ticipated responses (neither a correct nor incorrect 
answer to the problemi), the student receives an appro- 
priate feedback from the computer and is asked to 
answer the question again. The student's number of 
successive wrong answers is not changed. For example, 
a student might be asked the question "'Can you 
tell me how many different gametes could be produced 
by an individual whose genotype is AaBbCc?" The 
student responds '*yes'\ We consider such a response 
neither correct nor incorrect, and the computer would 
respond with something such as '*O.K., how many? 
Enter a number, please, to answer the question." 
In general, however, we have attempted to avoid 
ambiguous questions, such as the one in this example, 
by phrasing such questions as follows: *'How many 
different gametes would be produced by an individual 
whose genotype is AaBbCc?" 

Each problem allows for three unanticipated an- 
swers. The first unanticipated answer is considered to 
be incorrect. If the number of successive wrong answers 
is less than the wrong answer tolerance, the student 
is asked to try again; otherwise, he is forced off the 
course. The second unanticipated answer is not counted 
as being incorrect; however, the student is given a 
hint to the solution to the problem and asked to at- 
tempt the problem again. The third unanticipated 
answer is counted as being incorrect, and the student 
is told the correct answer to the problem, following 
which the number of successive wrong answers is 
checked to determine whether the student is sem 
on to the next problem in the module. 

After the student has completed the last problem 
rn ^he module, he is told how many problems he at- 
tempt<;d in the module together with the number 
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answered correctly. The student is then asked to 
sign off or to use the '^o to' command to take him to 
the next module on which he wishes to work. A student 
may repeat a module as often as he wishes. 

The following is an example of the output which 
would be produced at a terminal by a student working 
on one of the problems in the module labelled *men- 
dir*. 

One of the seven characters Mendel studied in peas 
concerned the shape of the pods — either full or 
constricted. The gene for full pods (F) was.found to 
be dominant over that for constricted pods (f). When 
homozygous FF plants were crossed to homozygous ff 
plants, the resulting Fl alt had full pods. When the 
F1 progeny were self ed {Fl x Fl), the F2 generation 
contained plants with either full or constricted pods. 

If Mendel scored 148 plants, how many plants would 
you expect to be phenotypically F? 

***** 48 

No. 

Please try again. 
75 

No. Herd's a hint. 

The genotypic ratio is 1:2:1. 
Please try again. 
***** calc .75*148 

111. 

***** 111 • 
That's right, Michael! 

Of these 148 plants in the F2 generation, how many 
would you expect to have the genotype FF? 

***** sign off 

LINE ISSIGNEDOFF, TURN OFF TERMINAL. 

In the example, the responses of the student are 
preceeded by This helps the student 

distinguish his typing from that produced by the com- 
puter. In the example, the student's first and second 



answers are incorrect. The student receives a hint after 
his second incorrect answer and is asked to try again. 
At this point, Ih^ student uses the 'caic* function to 
arrive at the correct answer. He enters 111 as his re- 
sponse and is told that he is correct. 

We have attempted to personalize the instruction 
by addressing each student by his name. The example 
ends with the student signing off *genpbs* before he 
answers the next quesiion. When he next signs on the 
course, the computer will resume his use of the course 
at the point where he signed off by repeating the second 
question in the example. 

A Coursewriter macro was written and used to 
generate the skeleton of a problem within a module. 
The skeleton, once generated at the appropriate 
position in the course, was then modified to insert 
the text of the problem, the specific wrong and anti- 
cipated answers, and the feedback which the student 
receives. The macro is designed to^permit teaching 
assistants and instructors not thoroughly familiar with 
the programming procedures of Coursewriter to con- 
struct problems for *genpbs' or to add new problems 
in the future with greater ease and dispatch. 

We have also written a' report module, a module to 
which only the instructor and his teaching associates ' 
have access. The report module allows the instructor 
to obtain an on-line summary of student performance 
on the problem modules. The report, which can be 
produced for one or a group of students registered 
on *genpbs\ will give the' name of the student, his 
*genpbs* registration number, the time spent on the 
course, the number of times a student has been forced 
off the computer for having exceeded the successive 
wrong answer tolerance, and, for each problem module, 
the number of problems attempted in the module 
and Che number of problems answered correctly. 
In the report, the problem modules have been num- 
bered 1 through 19 corresponding to the sequence 
of the modules as listed in Table 1. For example, the 
performance of a student on the problems in the mod- 
ule labelled "mendir* will be found in the report 
under module number 1; the performance of the 
student on the problems in the module labelled "link** 
will be found in the report under module number 4, 
etc. Figure 2 shows an example of the report produced 
for one student. 

The example of the report indicates that the student 
named John Smith, student number 1105, has spent 
a total of 848 minutes on the course. In module 6, 
the module labelled **genenv**, for example, this stu- 
dent has attempted 18 problems and answered 15 
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Today's Date Is 01/23/76 

Module 

Attempted 1 2 3 4 5 6 
Correct 

Name Number Time 
(c30) 

John Smith 1105 00848 53 23 16 26 53 18 
00003 40 21 12 23 43 15 

Figure 2. Instructor's report on an individual student 

correctly. The 3 below the student's name is the num- 
ber of times he has been forced off the course for 
having exceeded the successive wrong answer tolerance. 

A few miscellaneous features have been incor- 
porated into *genpbs'. A routine supplied by Central 
Computer Assisted Instruction has been used to create 
a portion of the course to which students do not have 
.?ccess, but which will allow the instructor to obtain 
an on-line summary of student use of *genpbs* at that 
moment. A second section to which students do not 
have access allows an erronous student record to be 
corrected. The latter module would be important if 
student performance data were used to contribute 
to a student's course grade. 



STUDENT USE AND EVALUATION 

The CAI course *genpbs' was first used by students 
enrolled in Genetics 500 at The Ohio State University 
Autumn Quarter 1975. In previous years this course 
utilized practice problems in the text book. Copies of 
the correct answers, with explanation, were put on 
reserve in the Biology Library. The course was taught 
by one instructor and two graduate teaching associates. 
There were 135 students who completed the course. 

Early in the quarter, a graduate teaching associate 
gave a brief, preliminary lecture to the students con- 
cerning the facilities available for their use (loca- 
tions of the terminals, hours CAI available, how to 
sign on *genpbs\ etc.). At this time, the students were 
given a handout similar to Table 1 and informed that 
they could begin work on the problems in the first 
module. The students were informed that the problems 
were available for their use on a strictly optional 
basis. The students w*ere not required to work the 
problems, and student performance on the problems 
did not contribute numerically to course grade. It was 
initially planned that a second lecture, devoted to ques- 
tions and difficulties which the students encountered 
during their initial sessions with 'genpbs', would be 
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needed. However, due to there being no such difficul- 
ties, the second lecture was cancelled. Throughout the 
quarter, as the instructor proceeded through the 
presentation of the lecture material, he would inform 
the students as to which modules in *genpbs' contained 
relevant problems. 

Six of the 135 students who completed Genetics 500 
elected not to use *genpbs'. The performance data 
for those students who used *genpbs' are given in Table 
2. In general, the mean number of problems attempted 
within each module is close to or exceeds the number 
of available problems. Students, therefore, repeated 
some problems, perhaps as a review effort. In any 
event, the data do indicate that students made consid- 
erable use of the problems. The percent correct data 
for some modules falls at or slightly below 60 percent. 
The problems in these modules are perhaps too diffi- 
cult for the students at the introductory level. Students 
spent an average of 15 hours working on *genpbs* 
throughout the quarter. 

In Table 2, performance data for three modules are 
absent since access to these modules was not allowed. 
Practice problems for these modules were handed out 
in class and answers were placed in the library on re- 
serve, as in previous years, in an attempt to compare 
student preference for the two types of problems, 
computer versus library. In the student evaluation 
of the course questionnaire distributed near the end 
of the course, 88% of the students indicated that, 
all things considered, they found the computer prob- 
lems more useful than the library problems. Ninety- 
three percent of the students indicated that they found 
the computer problems helped them understand the 
course material, while only 24% of the students indi- 
cated that they found the library problems helped them 
to understand the course material. _ 

For the 129 students who used 'genpbs* and com- 
pleted Genetics 500, the mean and standard error of 
the number of times students were forced off the course 
for having exceeded the successive wrong answer 
tolerance were 1.3 ±0.1 times. The greatest difficulty 
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Table 2. Student Performance Data on Problem Modules 



Name 


Module 
Number 


jNumDer ot 
Available 
Problems 


Number of 
Students 
Attempting 

At Least 
One Problem 


V 

Niiinher 
Attempted 


Mean ± Standard Error 

Answered 
Correctly 


Percent 
Correct 


mendll 


1 


28 


132 


32.5 10.91 


24.5 ±0.79 


74.8 ±0.89 


mendl2 


2 


23 


124 


26.2 + 0.62 


21.2 ±0.52 


80.6 ±1.06 


Stat 


3 


15 


121 


15.9 ±0.46 


11.6 X 0.40 ' 


72:7 ±1.35 


quant 


4 


26 


111 


24.5 ±0.58 


20.7 ±0.56 


82.6 ±1.42 


link 


5 


43 


126 


42.8 ±2.09 


32.9 ±1.82 


'73.6 ±1.45 


genev 


6 


17 


110 


18.4 ±0.72 


14.5 ±0.65 


72.1 ± 2.38 


chrathy 


7 


26 


n? 


28.3 ±0.69 


22.0 ± 0.64 


76.9 ±1.11 


tetani 


8 


21 


101 


21.7 ±0.64 


16.5 ±0.57 


75.2 ±1.25 


chmno 


9 


23 


0 








chmstr 


10 


26 


0 








prokar 


11 


35 


0 








bchemg 


12 


21 


77 


20.6 ±0.78 


14.7 +0.60 


71.2 ±1.49 


dnal 


13 


20 


112 


24.9 ±0.91 


17.1 ±0.72 


67.6+1.43 


dna2 


14 


25 


97 


24.9 ±0.84 


15.5 ±0.71 


60.3 ±1.70 


prosy n 


15 


22 


90 


22.8 ±0.76 


16.1 ±0.65 


70.0 ±1.42 


devgen 


16 


17 


90 


17.4 ±0.63 


10.4 ±0.49 


57.7 ±2.19 


nchmg 


17 


20 


84 


19.6 ±0.76 


12.0 ±0.63 


58.0 ±2.00 


popgnl 


18 


24 


74 


22.2 ±1.05 


15.6 ±0.90 


69.1 ±1.92 


popgn2 


19 


25 


59 


24.3 ±1.87 


16.1 ±1.45 


63.9 ±2.14 



with this feature was that students occasionally were 
forced off the course on evenings and weekends — 
times when they did not have access to the course 
instructor or course assistant in order to have further 
use of *genpbs* authorized. In addition, some students 
reported that other students were using their accounts. 
Such a student would discover when he attempted to 
sign on the computer that another individual had ex- 
ceeded the successive wrong answer tolerance and the 
student was unable to use his own account. We are 
considering two alternative plans to attempt to over- 
come this problem: One plan would be to increase- 
the successive wrong answer tolerance and warn the 
student when he had reached six successive wrong 
answers. The second plan under consideration is to 
leave the successive wrong answer tolerance at six, 
but to assign non-sequential sign on numbers in order 
to assure that one student cannot deduce the sign on 
number of another student. 

Student performance in the course was evaluated 
by two one- hour midterm examinations (100 points 
each), and a final examination (2 hour. 200 points). 
The examinations consisted principally of problems 
similar in kind and difficulty to those in *genpbs* and 
short answer essay questions. The course numerical 
grade of a student is obtained by summing the student's 



scores on the three examinations. We found a corre- 
lation of 0.26 between time spent on *genpbs* and 
course numerical grade. The correlation was found to 
be highly significantly different from zero. There may 
be a genuine cause and effect between time spent work- 
ing problems and course numerical grade; however, 
it is also possible that this observed correlation re- 
flects the tendency of good students to spend more time 
on optional material 'and the converse tendancy with 
respect to poor students. 

We^bmpared . student performance for Autumn 
Quarter 1975 with student performance in the preced- 
ing Autumn Quarter 1974 at which time 'genpbs* was 
not used. We found the mean course numerical grade 
to be slightly higher for Autumn Quarter 1975. 274.4 
out of 400 points versus 266.5; however, the difference 
was Jiot found to be significant. Use of *genpbs* is 
valuable to the student and at least as effective as other, 
traditional methods of providing problems and solu- 
tions to students (library problems). We feel the 
highly favorable student reaction justifies continued 
use of *genpbs*. 
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